DOCnHEHT BESOMS 



2B 201 993 



cs me 091 



IKaHTOTION 



SPmS AGENC: 
EO— DATE 
COIEEEACT 

3EER5 PRICE 
DTESCHTPTOES 



Bakerr Linda; Andersoiir Richsxd X. 
Effects of Inconsistent Xiifx)ziiaa±±ori on 
Processing: Evidence for Compile hen s i on i^anitoring . 
Technical Report No, 203* 

Bolt,- Beranek and Newman, Inc., Cambridge, Mass,; 
Illinois Univ., Urbana, Centex for the S-cudy of 
Reading. 

mtional Inst, of Educatinn (3D), Washington, D.C. 
Sriy 81 

a.00-76-01 16; N001 23-7 7-C06 22 
MF01/PC03 Plus Postage. 

♦ Cognitive P^rocesses: College Students; iHigher 
Education; *He±acogni tion; ^Reading Conrprehension; 
Reading Processes: *Seading Research; Reading Skills; 
♦Reliability o 
* Cohe r en c e ; * I ncnnsis t en c y 



fiS?::ffRCT 

Eixpository pas^^es containing either main point 
imrssistenci^s, detail inconsi33t:encies, or no inconsistencies were 
pr^^^TTted sHHisHie by sentence ^.-o 90 college students.. Subjects read 
thro.agh the :^s5SH:ges at;-:<:their aim pace and were encouraged to reread 
sectS^ns of tsxir: wlrenever they wished. As expected, subjects spent 
iirc!5.e time on ssarhsoces containing information that conflicted with 
izS:S::mation :pcsES3E£s5::ied elsewhere^ and they looked bacx more often at 
incmsistent :sent:sces. These .msdif ications in processing indicated 
tii3E±rthe subjects i^ionitored their comprehension as tie y were reading, 
actively evaluatir^r whether the ideas expressed in tie text were 
-crTttid.stent with ona: another. S^seral postreading measures provided 
:a5a!itional support ffor this conclusion. The relationship between 
^^eding behavior aisd subsequent identification of the inconsistencies 
wss:^ also examined^ iarge individual differences were found both in 
TKrcresslnq stratew^es and in confusion detection. (Author) 



* Reproductions supplied by EDRS arie the best that can be made * 

* from the original document. ^ * 

ERLC 



CENTER FOR THE STUDY OF READING 



U.S. DEPARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 
EDUCATIONAL RESOURCES INFORMATION 
.CENTER lERIC) 
^<^This cluciinturti has been loprodiicod as 
r»cmvtKl I torn ihu parson or or {)u nidation 
originiitiru) it. 
I : Minof changes Imvo been made lo irnprove 
teproduction quullty. 



Points ol view ot opinions stated in this docu- 
ment do not necessarily represent official NIH 
position or policy. 



Technical Report No. .203 

EFFECTS OF INCONSISTENT INFORMATION 
ON TEXT PROCESSING: 
EVIDENCE FOR COMPREHENSION MONITORING 



Linda Baker 

University of Maryland^; 
. Ba 1 1 i more County 



Richard I . Anderson 

Unl versl ty of 111 inol s 
at Urbana-Champalgn 



May 1981 



Unl versl ty of 1 1 1 inois 

at Urbana-Champaign 
51 Gerty Drive 
Champaign, Illinois 6l820 



Bolt Ueranek and Newma'ri Inc. 

50 Moulton Street 

Cambridge, Massachusetts 02238 



The research reported herein was supported in part by the National Insti- 
tute of Education under Contract No. HEW-NIE-C-AOO-76-OII6 to the Center 
for the Study of Reading, in part by the Advanced Research Projects Agency 
under Contract No. N00123-77-C-0.622 , and in part by a UMBO Summer Faculty 
Fellowship award to the first author. A preliminary report of this 
research was presented at the meeting of , the American Educational Research 
Association, April 1979- We wpuld like to express our appreciation to 
Sally Standiford for her help in the'pilot phases of this research and to 
Deah Radln for his assistance in programming the computer and running 
subjects. We would also like to thank Bruce Lombardi and Susan Sorinenschein 
for their comments on earlier versions of this manuscript. 



Bf7:TRIAL BOARD 

.i.^er,.z3rr:3ton, Chal person 

Roberta-Eerrars Jim Mosenthal 

Scott Fert^ii Ann Myers 

N i cho 1 as Basil ngis Andee Rub i n 
Asghar Ira^Hb:^^ ' Wl 1 1 i am Ti rre 
Jill LaZ3^"^% ^ Paul Wilson 
Peter Wlnograd 

Mi fvfr/ens, Editorial Assistant 



. Process inxi inconsistencies 
1 

Abstract 

Expos i tor-'passraxBt-s ^orrtaining either main point incor^ stencies . detail 
i neon sist:enc Fes, o inconsistencies were presented HHsntence? by sentenet^ 

to 90 college stud' nrt^. Subjects . read through the passages at their own 
pace and were gn.. ^uprgied to reread previous sections of text whenever the 
wished. As e<pecired, subjects spent more time on sentences containing 
information :t&a±:conf^ with information presented elsewhere, and th:r.~^ 

lool<ed backncsre often at inconsistent sentences. These modifications 
processing indicate that the subjects monitored their comprehension as 
were' reading.,, actively evaluating whether the ideas e>-^>>'essed in the te: : 
were consistent wi th one csnother. Several postreading measures provide, 
additional support for this conclusion. The relationship between readr-r^ri 
behavior^d subsequent i denti f ication of the ihconsislrencies was also 
examined. Large individual di f ferences were found Doth in processing 
strategies and in confusion detection. 
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Effects of biCTnsi stent Inforrnation on Text Processing: 
Evi^2Ence for Compre+:sns ion Monitoring 

The continuing 3=:rgi3 of interest: the cognitive processes involved 
in comprehension has giv«n rise to a domain of inquiry: the role of 
miitacognition in comprehension (Baker. 1979c; Baker & Brown, in press;. 
Brown,- I98O; Flavell, in press; Markmsn, in press). Metacogni tion refers 
to one's knowledge and control of his own cognitive processes (Flavell, 
1S78). The metacogni tive activities involved in comprehension include 
keeping track of the success with which onels comprehension is proceeding 
and ensuring that the process continues smoothly by taking remedial action 
if comprehension falters, 

it has long been argued that these monitoring activities are crucial 
to effective reading^^(Dewey, 1910; Hu3y, 1908/1968; Thorndike, 1917). and 
many recent theories of comprehension incorporate monitoring components int( 
their models (e.g., Collins, Brown, & Larkin, I98O; Goodman, 1976; Just & 
Carpenter, 1980; Ruddel 1 , 1376; Rumelhart, 1 98O) . We now have ample 
evidence, that beginning and less able readers appear to be deficient at 
evaluating their understanding of text (Baker & Brown, in press; Di Vesta, 
Hayward, & Orlando, 1979; Garner, 1 98O; , Markman , 1979; Winograd & Johnston, 
1980; Forrest 6 Waller, Note 1; Paris & Myers, Note 2) , but there have 
been few empirical tests of the crucial assumption that mature readers do 
monitor their comprehension effectively. Baker (1979a) conducted a pre- 
liminary study to ascertain whether this assumptvon is warranted. The 
study employed a "disruption" paradigm (Markman, 1977; Miller & Isakson, 
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1978), wn&rsin conf us i ng el ements are deliberately introduced into a text 
and fai'iunss to notice the disruptions are taken as evidence of ineffective 
comprre hens ion monitoring. 

College students were presented with s i x . expos i tory passages, each 
containincr either an inappropriate logical connective, an ambiguous referent, 
or an iimccns i stent fact. They were instructed to read the passages careful ly 
in preparatfon for subsequent "discussion" questions without being told 
that cifsruptions were present. After reading, the subjects answered ques- 
^'tions requiring recall of the deficient sections of text. The purpose of 
the recall task was to reveal whether subjects modified the disruptions 
in seme way to render them more sensible. Next, subjects were informed 
that the paragraphs did, indeed, contain confusing sections and were asked 
^ to identify them, rereading the paragraphs if necessary. The subjects were 
also asked to comment retrospectively on how they had reacted to the con- 
fusions when they were first encountered and how the confusions had affected 
their understanding of the passage. 

Across all subjects and passages^ only 38% of the confusions were 
correctly identified. Though this figure suggests that subject? were poor 
at monitoring the i r comprehension, the recall protocols and retrospective 
reports revealed otherwise. The subjects had a wide repertoire of strategies 
availabie'f^ confusing sections of the text. (See Baker, 

1979b, for a detal led discussion of these strategies.) For example, they 
made inferences that resolved the conf us ions ; they reread the passage or 
looked ahead In search of clarification, they decided the confusions were 
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too trivial to attempt to resolve, or they assigned an interpretation to 
the text that differed from the intended meaning. In short, most students 
apparently did evaluate and regulate their understanding, even if they did 
not perceive the intended disruptions as such. 

One limitation of the Baker (1979a) study is that.it relies on data 
obtained after reading to make inferences about events occurring during 
reading (Ryan, in press ; S imons , 1970- The evidence for ongoing compre- 
hension monitoring would be more compelling if we found on-line modifica- 
tions in text processing due to the presence of a disruption. The present 
study was designed in an effort to provide such evidence. 

Passages were presented on. a computer terminal, sentence by sentence, 

.J i 

under the i ndi vi duaT reader ' s control. Subjects advanced to subsequent- 

■ I - 

sentences at their own pace and were encouraged to look back at previous 
sentences whenever they wished. The computer automatically recorded the 
amount of time each sentence was exposed and the pattern of movement 
through the text. Each subject read three experimental passages, one with 
.an inconsistent main idea, one with an inconsistent detail, and one with no 
i.nconsistency. It was expected that subjects would spend more time reading 
a particular sentence when it. contained information inconsistent wUh the 
passage than when it was consi stent. It was also expected that suj)jects 
would look back more often at the inconsistent sentences in an effort to 
resolve or verify the problem. After reading each .passage , subjects were 
asked to answer several multiple choice questions in order to reveal how 
the inconsistent information had been interpreted. Finally, confusion 
detection was assessed directly by asking subjects to indicate which 
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sentence, if any, contained an inconsistency and to report whether they 
noticed the problem during their initial reading of the passage. 

Prior to reading, half of the subjects were informed that inconsistent 
information would be present and that- they would be asked to identify the. 
inconsistencies later. Alerting the subjects in this way should put them 
in an editorial processing mode, thereby increasing the amount of time 
spent on the passages and increasing the likelihood of noticing the incon- 
sistencies during reading. 

A secondary goal of the present study was to attempt to develop a 
profile of the successful comprehension monitor. Although our subject 
population is comprised of individuals who read at or near college level, 
there are undoubtedly large individual differences in thoir reading pro- 
ficiency. Therefore, we expect that some students will be more sensitive 
to-the inconsistencies than others. The question of interest is whether 
there are specific processing activities characteristic of subjects who 
subsequently report the inconsistencies that are not characteristic of 
the nondetecting subjects. 

Method 

Material s • . - ■ ' ■ ' 

The materials consisted of four three-paragraph passages that dealt 
with topics in world history. The passages, written by the experimenters, 
were based on C 1 i ff ^ s Course Outline in World Civilization {Leon, 1970). 
Each of the three paragraphs in a passage focused on a separate aspect of 
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the main topic. For example, in a passage about the Moslem civilization, 
the first paragraph dealt^wKh^ the social structure, the second with the 
economy,— and the third with the culture. One of the four passages served 
as a warm-up passage and always appeared in its original form. The middle 
paragraph of each of the remaining passages was modified to contain incon- 
sistencies involving the, main idea of the paragraph and a detail. Only 
one of these inconsistencies appearedjn a given passage at one time. 
The inconsistencies were created by. replacing a single noun or adjective 
with a word that conveyed an opposite or incompatible meaning. 

All passages were similar in length and each paragraph contained five 
sentences. Since the middle paragraph was of primary concern in t>-e -'X.>eri- 
ment, Its structure was more carefully controlled across passage: . V ; M-st 
and fourth sentences of the paragraph .contained the main point and Cc;t?.! ! 
inconsistencies, respectively. Both of these sentences wsre 10 words In 
length and contained 19-21 syllables. The inconsistent„words that appeared 
in the modified versions of th- paragraphs were similar in number of letters 
and syllables to their-consistent counterparts. The remaining three sentences 

contained l^t-lS words each and had comparable numbers of syllables across 

■ - . > - . 

passages. 

In addition, the organizational structure of the middle paragraph was 
the same across passages. The opening sentence introduced the main topic 
of the paragraph, and the four subsequent sentences each provided a support- 
ing point. Thus, in the main point inconsistency condition, the first 
sentence of the paragraph conflicted with all other sentences. The detail 
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inconsistency was embedded within a supporting point and was not in itself 
related to the topic sentence. It was always based on information contained 
in the immediately preceding sentence. 

Insert Table 1 about here. 

/ 

One of the paragraphs used in the experiment is presented in Table 1 
in its consistent version with the words that were substituted in the 
inconsistent" versions indicated in parentheses. The main idea inconsistency 
appears in the first sentence, where the economy is said to be characterized 
/by poverty. But subsequent sentences describe positive aspects of the 
economy: control of the trade routes, successful industries, flourishing 
agriculture. The detai 1 -1 evel inconsistency is in the fourth sentence , 
where it is stated that the textile industry was government owned. But 
the previcus sentence said that all of the industries were privately owned 
and .operated. In order to notice either of these inconsistencies, a reader 
would need to integrate information across sentences. However, the state- 
ment of the. main idea conf 1 i cted wi th virtually all of the other sentences 
in the paragraph, while the detail statement conf 1 i cted with only one. 

Six multiple-choice questions were constructed for each passage, two 

based on each paragraph. Each question' asked d subject to select an appro- 
priate paraphrase of a segment of the text. The questions on the first 
and third paragraphs were included to prevent subjects from focusing exclus- 
ively on the middle paragraph and wi 1 1 not be considered further. One of. 
the questions on the middle paragraph involved the main idea and the other 
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involved the detail. Each question had four alternatives, one of which 
was a paraphrase of the consistent version of the statement and another 
of the incoVlsi stent (see Table 1 for examples). In addition, the option 
"cannot be answered on the basis of the paragraph" was provided for sub- 
jects who noticed the inconsistency and were unable or unwilling to 
resolve it. . , 

Design ^ ' j 

■ ' The experimental design consisted, of three within-subjects factors 
and one between-subjects factor. The within-subjects factors were passage 
(Inca, Moslem, and Byzantine), type of inconsistency (main point, detail, 
and none), and order In which a particular passage was read (first, 
second, or third). The between-subjects factor was whether or not 
subjects were alerted before reading that the passages contained in- 
consistencies. A Latin Square design was used for counterbalancing the 

three within-subjects factors. This design, described in W,lner (1971, 

3 

pp. 739-7^5), uses all 27 treatment combinations of a full 3 factorial, 
but each subject proviues only three observations. Thus, each subject 
read each passage, but the order in which the passage was presented,- 
and the type, of disruption it contained, depended on to which of the 
nine presentation groups the subject was assigned. 

Subjects in the experiment were 90 undt^graduates enrolled, in an 
educational psychology course at the University of Illinois. The majority 
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of the students were sophomores and juniors. It was found after the 
experiment was conducted that the data sets for three subjects (from 
different cells) had been lost. Since it was no longer possible to run 
additional subjects, and because the analysis of var i ance outl i ned by 
Winer requires equal cell size, the .missing data were replaced by cell 
means. 

Procedure • . 

The enti re experiment was controlled by the PLATO computer system.^ 
Subjects were run in several large groups in a classroom containing 
30 PLATO terminals, each with an a.l pha-numeri c keyboard and a plasma 
^.display screen. An ar.si stant /was present during each session to instruct 
subject-^ how to sign onto the computer and to answer questions. As 
subjects signed on, they were automatically assigned to one of the 18 
cells in the design according to a predetermined plan. 

All instructions for the experinient were presented via computer. 
The instructions were displayed sentence by sentence, one paragraph to 
a "page" of screen . Each sentence appeared underneath the immediately 
preceding one so that, in effect, subjects read down the screen. The 
rationale of having the display position of sentences move down the screen 
was to provide a'spatial cue for each sentence's posl ti on • i n the paragraph, 
'thereby mai<ing it easier ^for subjects to re- expose "a particular -sentence. 
As each new sentence was presented, the previous one was erased. Sub- 
jects were instructed to initiate presentation of each sentence by pressing 
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a key labeled *'NEXT.** They were told they could look back at the pre- 
ceding sentence by pressing a key labeled "BACKJ' They coufd reread 
the entire paragraph by pressing a key labeled '^LAB," which took them 
directly back to the top of the L-creen and was rrore convenient than 
repeatedly pressing the BACK key. It was recommended that, in order 

-to facilitate movement through the text, subjects use the three middle 
fingers of their right hand to press the NEXT\, BACK, and LAB keys, 
respectively. The computer program provided subjects with ample 
opportunity to familiarize themselves wjth the various control keys 
and to prap^t^ici noving through the text as they read the instructions. 
Further InsthMctlons informed subjects^^ that they would be reading 

.passages dealing with topics in world history and that they shoul d 

read them carefully I n preparation for , subsequent questions on their 

content. No mention was made of the fact that- the amount of time spent 

on each sentence 'woul d be recorded. Half of the subjects received the 

following additional instructions:." 

Some of the 'paracjraphs that you will be reading contain in- 
consistent information, where ideas expressed in one sentence 
conflict with ideas expressed in one or more other sentences. 
^ . . Carefully evaluate the information in each paragraph for in- 
consistencies. Make a mental note of any conflicts or contra- 
^' dictions you detect. You will get questions about them later. 
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When subjects finished reading the instructions, they went on with 
the experiment at their own pace. The passages were presented in the 
same format as the instructions, one sentence at a time, one paragraph 
to a page. Subjects could reread each paragraph as often as they 
wished, but once they pressed the NEXT key to go on to the next para- 
graph, they could not return to a previous one. The warm-up passage 
was presented first, followed by the three experimental passages. 
After reading each passage, subjects answered the six multiple-choice 
questions by pressing the appropriately numbered key. The computer 
recorded the responses and the amount of time' taken to respond. 
^ . When subject's had completed the reading and question-answering 
tabks, they were given instructions for the detection ,task. They were 
informed that inconsistencies had been present in some of, the passages 
(for half of the subjects, this was actual ly. a reminder) and that they 
would now be. asked to identify them. They were shown the middle 
paragraph from each of the four passages, wi.th the sentences numberedv 
1-5. Subjects were asked to indicate the number of . the sentence. they 
thought ^contained the inconsistency arid to select *'6»' if they thought 
the paragraph did not contain ari Inconsistency. If subjects indicated i 
that a'confusion had been present, they were automatically branched 
to two additional questions: (a) Did the confusion involve a main 
idea or a detai 1 and (b) Did you! notice the inconsistency during the 
Jnitlal reading of the passage?' Upon completion of the detection task, 
a brief explanation of the purpose, of the experiment was provided. 
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Results 

-This section of the paper is divided into three sections. The 
first section reports the data obtained from the two processing measures, 
exposure times and. number of re-exposures of target sentences. The 
second section examines the responses and response times to the questions 
based on the main Idea and detail target sentences. The final section 
describes the detection data and includes a brief report of a study 
carried out to validate the detection measure used in the present 
experiment. Consideration of the relationships among the- dependent . 
measures will be postponed until the Discussion section. 

In order to eliminate redundancy, we wljl first make several general 
.comments which pertain to all analyses. First, the order in wl.ich a 
.particular passage was , presented was entered into each analysis as a 
factor. Although one. passage was presented as a warm-up passage, one 
might still expect, to find changes in performance as subjects warmed up 
still further to the task" and became more, accustomed to moving. around 
through the text, responding to comprehension questions, and searching 
for inconsistencies. Many of the. analyses di d, in fact, reveal a 
reliable effect of order whi ch. was undoubtedly due. to such practice 
effects. However, the main effect of interest, type of inconsistency., 
was counterbalanced across order, and interactions involving order were 
either not reliable or were uni nterpretabl e. To simplify exposition of 
the results, we will not report the order effects in the text. The 
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interested reader may consult Appendix A for^a listinc of all significant 
order effects and Appendix B for a discussion of the order effects and 
a listing of the cell means which show main effects of order and inter- 
actions of order with type of inconsistency. 

Second, the passages which served as the experimental materials were 
entered into the analyses as fixed effects.. Though we attempted to make 
the passages comparable on structural, thematic, and organizational 
factors, they did, of necessity differ in specific content and, there- 
fore, perhaps In familiarity and comp rehensi bi 1 i ty . Hence, it would 
not be surprising if the passages differed in the salience of their 
inconsistencies or in the difficulty pf questions based on ^hem. Several 
analyses showed . re 1 i'able effects due to passage J but these effects will " 
not be considered in the text for the same" reasons mentioned in the 
preceding paragraph. (See Apperrdlx A for a listing. of reliable passage 
effects , ' . -''^ ' ' ^ . '' ^ 

Third,^ because instructing subjects to be on the alert for incon- 
sistencies did not affect performance on any of the dependent, measures, 
the analyses have been collapsed over the Instruction factor. Prel imi.hary 
analyses reveal eel that al erted subj ects did not spend more time reading 
the pas sageTor look back at criticar sentences more often than unalerted' 
subjects, nor we re. they more likely to notice the inconsistencies during 
.reading. One possible reason for this 1 ack of an effect of Instructions, 
is that .the demand characteristics of the task were such that all subjects 
processed the text careful ly i n preparation for the test questions. 
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All data analyses follow Winer's Plan 11 analysis of variance for 
Latin Squares (1971, pp. 739-7^5) with order, passage, and type of 
disruption as wl thi n-subjects factors. Note that level of idea (main 
point vs. detail) does not appear as a factor in the design., Since 
these ideas involved qualitatively different kinds of information and 
appeared in different serial positions, we would not know to^which of 
these differences an effect was attributable. Therefore, for each de- 
pendent measure, two separate- analyses were carried out, one" for the 
mai^n idea ancj another for the detail. The question of primary interest 
i,s whether the type of Inconsistency present in a pasisag^e (main* poirrrc, 
detail, none) affected performance. For the maiti idea, the critical 
"comparison was ^between the inconsistent main point condi tion and the two^ 
consistent main point conditions.- For the detail; the comparison was 
between the inconsistent detail condition and the two consistent' detai 1 
cohdi tiohs. . .. ' > 

When a reliable main effect of type of inconsistency was obtained, 
Fisher*s Isd procedure was used to determine the locus of the effect. 
Unless otherwise noted, the twc consistent /condi tions were always reli- 
ably different from the i nconsis:tent condition and were not reliably 
different from one another. The rejection region for all statistical ^ 
tests was £ < .05. , . 

Processing Measures . ^ ; 

Exposure times . The computer automatical ly recorded the amount of 
time each sentence was exposed on the screen. Exposure time is not ' 
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necessarily mous with reading time, but it provides an indication 

of the amour time spent readi ng, studying, or thinking about the 
sentence. Exposure times were obtained for every sentence in every 
passage, but the data of concern involve only the middle paragraph of 
the three experimental passages. There are a number of possible ways 
to analyze these data, but the dependent measure we adopted was the tptal 
amount of time a subject spent on the second and fourth sentences. ^ 
The reason for focusing on the fourth sentence is obvious: : It corjjtains 
the word that i s i nconsi stent wi th i nforma'tion .proyi ded I n the' thtj-d 
sentence. The reason for focusing on the second sentence may be less 
'"apparent: because the. f i rst sentenceactual ly conta fned 'the inconsistent 
information. However, there Is no reason to expect differences in 
reading time unti 1 subjects read beyond the manipulated sentence and 
encounter conflicting i nformation. An analysis of .time spent' on the 
first sentence supported this^siumpti.on The reason for analyzing • 
total exposure time is that/pilot Work revealed it t.o.:be a more appro- 
priate measure than time, on first exposure;, given .tha±: Individual 
subjects varied greatly /in the ir approach to the task. Some read 
through the. parag raph very rapidly, -then went back to the >egl nning 
to read each sentence more slowly. .^Others spent a great deal qf time , 
on each sentence on their first and only exposure to W. Pilot work;:, 
also indicated; that time on (the entire paragraph was not an appropriate 
measure because excessive variance was cbnt ri buted by the other sentences 
In the paragraph. , ,, ' 
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Insert Tab] e._2 about here. 

The mean total exposure times on the main point and detail target 
sentences (sentences 2 and A) are presented on the left side of Table 2, 

where it can be seen that our expectation of longer exposure times on 
inconsistent sentences was upheld. Analysis of variance of the main 
point exposure times y iel ded a rel iabl e effect of -type of, i neons i stency, 
such that subjects reading passages containing a main point inconsistency 
spent more time on the second sentence than subjects reading the passages 
containing either a detail inconsistency or none, F_(2,162) = 31-91. 
The exposure time analysis for the detai 1 target statement also revealed 
■a .reliable effect of inconsistency type, such that subjects encountering 
a 'detail inconsi stency spent more time on the target sentence than did 
subjects not encountering^^^a detai 1. inconsistency, £(2, 162) = 25.78. In 
addition, application of Fisher's l.sd procedure indicated that subjects 
who encountered 3^;^ma in point inconsistency also spent more time. on the 
detail statement than subjects- reading normal versions of the passages, ; 
suggesting that the main point inconsistency. led to, slower reading , of 

■ the entire paragraph. Though one may argue that the differences on 

"the detail target sentence arose because the inconsistent word was 
somehow more difficult than the consi stent. word it replaced , /this 

-argument is untenable for the main. point results. The difference due ; 
.to a main point incons i stencyappeared on subsequent sentences whiqh 
were identical for all subjects, . j • 
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Re-exposures of target sentences . The second on-line dependent 
measure was the number of times subjects initiated additional exposures 
of the target sentence after their first exposure to it. The question 
of; interest is whether subjects were more likely to go back to the 
target sentence when it contained an inconsistency than when it did not. 
The right side of Table 2 shows the mean number of re-exposures of the 
main point and detail target sentences.. The main. point analysis 
yielded a reliable effect of type of inconsistency, £(2,l62) = 6.07. 
Sentences involving inconsistent mai n points were re-exposed signifi- 
cantly more often than sentences involving consi stent ma i n poi nts . The 
analysis for detail sentences failed to reveal a. statistical ly reliable, 

^'difference due. to type of inconsistency; despite a trend in the expected 
direction, T(2, 162) = 2.33, £ = ..10. , ' . - 

The: analysis pf re-exposures provides further evidence that subjects 
noticed the inconsistencies during reading, but it does not reveal what 
processing strategics were used/ 'Were subjects more likely to reread 
the enti re -paragraph when they encountered an inconsistency or did ; 

"they reread the previous sentence immediately after reading the incon- 
sistent sentence.? An effective monitoring strategy for subjects who. 

■ ' ■' " ' ■ ■ ■ 

noticed a problem would be to. go back to the preceding sentence to 
verify that som.e thing was- i neons is tent. ' : " 

1n .order to determine^ whether this strategy was used, the subjects' 
data records were examined for the fol 1 owing, patterns of sentence exposure. 
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1,2,1 and 3,4,'3. The number of subjects engaging in this strategy was 
low, but there were clear differences due to inconsistency condition. 
Upon exposure to a main point inconsistency, 3\% of the subjects reread 
the initial sentence immediately after reading the second. In contrast, 
If the main point was^'consi stent, only" 7.5^ of the subjects used this 
pattern of inspection. Similarly, 16^ of the subjects encountering an 
inconsistent detail looked back. at the preceding sentence, whilelonly 
S% of the subjects did so for a consistent detail. The apparent 
difference in the number of subjects who ^looked back at main point and 
detail inconsistencies is provocative, though it m^y well be an effect 
due to serial position in the paragraph. - 

- . ■ - \ * . " ' . "tj ^ ' ' 

Responses to Questidns Based on Main Ideas and Details , : 

Subjects;' responses to the two multiple-choice questions of interest • 

from each passage were categorized as b^elongi.ng to one^of four di fferent ' 

■response classes: (a) consi stent--the alternative selected was compatibFe 

with the consistent version of the main,. idea or detail; (b) inconsistent-- 

the ■al ternative selecte^d was compatible with the ,i nconsi stent version of 

-the main /idea or detail.; (c) "can't answe r"r-the alternative selected', 

stated that the question could not be answered on the basis-of the 

'paragraph; and (d) other--the alternative selected was one of the two' 

■■• - ' ■ ■ ' ' - - ■ ' '•■•'■'. 

distractor i,tems. The proportion of responses falling into each.of ^, 

these categpries for the main point and detail questions is presented ' 

in , Table 3 as a function of the type of : incons i stency present in tkis passage 
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Insert Table 3 about here. 

Arivilyses of the dq^a summarized injlne 1 of Table 3 revealed that 
'Consistent" responses were Tess likely when the questions were based 
on Inconsistent Ideas than when they were based on consl stent 1 deas 
(F^(2,l62) = 61.03 for main point questions; F^(2,l62) - ^5.^1 for detail 
questions). Such ap outcome Is hardly surprl s 1 ng , for a "consistent" 
response to a quest ion based on Inconsistent, Information would only 
occur If subjects. had adopted the consistent Interpretation, either by- 
resolving the inconsistency or drawing an inference based on other' 
Information in the passage. Though about one-thl rd of the subjects 
apparently did so,' a comparabVe proportion, of the responses fell into 
the 'Hnconsistent" category A subject, might select the inconsistent 
^alternative for either of two reasons: (aj he did riot notice that 
the target statement con fl Icted wl'th other Information in the paragraph; 
or, (b) he noticed thel inconsistency , but , i n compliance wi th the per- 
'ceiVed 'task demands, .he responded wtth the alternative that was compatibl 
.with the .information he -'had read. Table 3 also shows that the "can't . 
.answer", option was selectecb somewhat more often when an inconsistency - 
was' present, bu§ the difference iri frequency was not reliable. This , 
option was' Intended to" provide the means for subjects ;^^to indicate they 
were aware of an Inconsistency, but the wording may have been too vague 
for subjects. to realize Its Intended use. , ' 
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The variability in the responses to the questions based on Incon- 
sistent information suggests that comprehension of the material was 
disrupted. More compelling evidence,^ however, is provided by the 
response time data which, are summarized In l;|ne 5 of Table 3. Analyses 
of variance were carried out on the-amount of time required to ansvjcr 
the main point and detail questions, regardless of which response was 
selected. Subjects consistently required mo rje time to answer both types 
of questions when 1: hey: we re based on Inconsistent rather than consistent 
information, F;(2, l62) = 10.55 and F(2,l62) = 3. 76,for' main point and ^ 
detail quest ions , respectively. j - 

InconsI stgncy Ident i f Icatlon , . ; ; , ' 

V In the last part of the experiment , subjects were provided with the 
middle: paragraph of each passage and were asked to Indicate which sentence 
.if any, contained, an Inconsistency. Responses were scored-as correct. 
identifications as follows: rpaln point Incons i stency, J Ine 1 ; detail 
inconsistency^ .line k; no incons 1 stency , 1 1 ne 6: Across all subjects 
and passages, ^approximately two-thirds of the line numbers were correct. 
As shown .in Table the proportion of I dent I f I cations was similar across 
inconsistency conditions^, and an analysi s of variance confirmed that the 
differences were not statistically reliable (F^< 1). 

ln$eft Table ^ about here. ^ 
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V a i I da t ion study . Because the computer was not programmed to allow 
subjects to explain the nature of the snconsi stency they detected, it 
is possible that some subjects who failed to give the appropriate line 
number actuplly did detect the inconsistency. It is also possible thdt 
some subjects who provided the appropriate line number did not pick up 
the inconsistency we intended to convey. In order to validate this 

measure of confusion detection, a second study was carried out. 

"*■'■■■■ " . ' - /' ♦ 

In this study, 36 University of Maryland Bartimore County under- 
graduates were provided wi th the middle paragraph of each passage, typ6d 
on separate sheets of paper. The order of presentation was counter- 
balanced using the same Latin Square design described earlier. :. The 
subjects were informed, that some of the paragraphs contained a fact^ 
that was inconsistent with other facts in the paragraph and that the 
inconsistency could i nvol ve ei ther "a :nTa in idea -or a detail. The subjects 
were first asked to' :cl rcl e the line number of the sentence they thought 
contained an incons i stency , ci rcling line 6 if no incons i stency , was 
present, they were .then asked to unde r1 ine the^'word or, phrase that 
was iTiost inconsistent arid to explain what it was about the sentence 
.thait made it inconsistent. 

TKe proportion of correct identifications was first calculated only 
on the basis. of the line numbers given.. These,data appear in line, 2 

of Table where jt can be seen that . the.detection rates for the main 

■ > ■ . \ ■ 

poTnt and detail inconsistencies., are similar-^ to those of the main 
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experiment. However, subjects were rrnxh less likely to indicate that 

no inconsistencies were present in the consistent passages.^ The reason 
for this discrepancy is not clear; perhaps the demand characteristics 
of the task, in which written explanations were required, induced 
subjects to read more carefully and to use more stringent criteria 
for evaluating consistency, (Remember, too, that a different popula- 
tion of students was samplec|,) 

• o . . . ' 

' . -■ » 

The ver.bal explanations were then examined for evidence that^the 
jn tented inconsistenci es had been- identi f ied.. The detection rates 
based on" these, explanations shown in line'3 of Table ^, are similar ' 
to those based on the lihe numbers;. Of most importance is that, with , 
only four exceptions, there v/as a perfect correspondence between the 
line n.umberthe subject indicated as conta ining^ the incohsistency and 
the' verbal explanation; Therefore, we may conclude that the data from 
the original experiment, dp provide a , sensi 1 1 ve index of; inconsistency 
detection. ; ^ ^ 

Self-reports of detectibn during: readi ng . ' Returning now to a con 
sideration of the. original experiment, i^f a subject reported that an 
inconsistency was present, the comp^uter program autpmatical ly branched 
to an additional question asking whether the inconsistency was" noticed 
during the ini t ial .readi ng of the passage. Of those subjects who pro- 
vlded'the correct line numbers for the main.point i neons i stency , ._64| 
reported noticing it during reading* Simi 1 arly, of those subjects who 
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Identified the 1 ! ne containi ng the detail 'inconsistency, 63% reported 
noticj;ing it duri ng readi ng . Though self-reports are often suspect, 
the fact that reading times and re-exposures of target sentences were . 
affected by the presence of an inconsistency lends credence to the 
subjects' claims. More direct evidence of a relationship between these 
sei f-rejjorts and processing behavior is provided by those subjects 
who adopted the strategy of immediately re-exposing the sentence involved 
•in setting up the inconsistency (i .e., sentences 1 or 3) . Of those sub- 
jects who.: used this inspection pattern and detected thenain point 
inconsistency, 9^% reported noticing the i neons i stency duri ng reading. 
Similarly, 92% of the detecting subjects who. went baci< to ,,the sentence 
preceding the detail inconsistency reported noticing the problem during 

..reading. ; (,\ ' " ' i. 

The. fact that the main point and "detail i neons i stenci ^\ were apparently 
equal'ly salient to subjects duri nd readirfg was somewhat surprising. , How- 
ever, a plausible .explanation' comes from responses to a secpnd? question 
asi<ed of subjects who-^repor ted air incpnsi Stency: Does the confusion = 
involve a ma|n, i dea or a detai 1? •Considering only those sybjectswho \ 
provided the correct 1 i ne numbers^, 9U correctly, identified the main ■ 
Idea Vincorisistencies as such, but also' classified thedetail i n- 
consistencies as main ideas. This suggests that subjects may have, devoted 
equaV attention to the main idea and detail statements because they 
considered them "of equel importance.. Another ' poss i bij ity is that "by 
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virtue of the demand characteristics of the task, all Information was 
processed to the same extent, 

^ >i Pi scussi on 

The primary purpose of the present experiment was to obtain on-line 
evidence of comprehension monitoring during reading. To this end, the 
experiment was successful • Students spent more time on sentences con- 
taining inconsistencies, and they looked back at them more often than 
.at consistent sentences. These modifications In proces'slng provide 
compelling evidence that mature readers evaluate and regulate the success 
of .their ongoing effot-ts to comprehend. The data also are consistent 
with current models of reading which view comprehension as an active 
constructive process Ce.g., Just & Carpenter, 1980; Rumelhart, I98O). 
Recent empirical tests of such models have provided additional on-line 
evidence of changes In reading behavior. For example, readers return 
to previously read information and make regressive eye movements when 
they encounter pronouns whose referents are unclear (Carpenter & Just, 
1977; Garrod S Sanford, 1977-), and they require more time to read 
paragraphs which violate conventional organizational structure (Greeno S 
Noreen, 197^; Kieras, 1978). 

~7 The available evidence suggests that the construction of meaning 
typically^ proceeds smoothly and relati vely automati cal ly, but If a 
difficulty Is experienced, the. reader slows down and allocates extra 

'attention, to the problem area (Brown, 1 98O) • The reader may deploy 
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debugging strategies suth as making inferences, rereading, or jumping 
ahead In search of clarification. One of the strategies used for 
debugging purposes In the present experlrnent was the Immediate re- 
exposure strategy discussed previously. After evaluating their under- 
standing and finding it Inadequate, subjects attempted to remediate the 
problem by checking to see If they had correctly understood the previous 
sentence. We must not conclude, however, that subjects who did not use 
this strategy were deficient In moni tori ng their comprehension. Rather, 
these subjects may still have had the propositions from the previous 
sentence stored In working memory (Kintsch S van Dljk,- 1978) , and so 
they checked memory Instead of the** text Itself. An Interesting empirical 
question emerges from the recent demonstration by Daneman and Carpenter 
(1980) that better readers Jjjjjive a larger working memory capacity than 
poorer readers. Perhaps It Is only the less able readers. who, upon 
encountering Information that seems Inconsistent, must physically 
reinstate the previous sentence for verification. 

The fact that the majority of subjects were sensitive to Inconsistent 
Information, suggests that they evaluated their understanding by testing 
whether the Ideas expressed In the text were consistent with one another 
(Baker, 1979b), This Internal consistency standard seems to' be applied 
.spontaneously given that Instructions to evaluate\the text for consistency 
did not differentially affect performance. There Is also evidence- that 
: students use Internal consistency at the expense of other appropriate 
ievaluatlon criteria. Pgr example. Baker (1979a) found that when , . 

a'.;' ' V " . ■ 28-, ' .■ , ^ ■ 

er|c 
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inappropriate logical connectives were introduced into expos i tory passages 

subjects did not identify them as confusions, but reported that the ideas 
expressed in. the two clauses, were inconsistent with one another. 

"^While it may be true that evaluating text for consi stency is the 
default standard for readers who are moni tori ng thei r comprehension, 
what of the students In the present „study who did not identify the 
intended inconsistencies? Though we have argued that failure to report 
an inconsistency is not necessarily evidence of failure to monitor compre- 
hension, the fact that one third of the college students In our sample 
failed to report the inconsistencies is troublesome. ^ One coul^d invoke 
Grice's (1975) cooperative principle to explain why students did not 
notice the disruptions during reading: They expected the writer to state 
only what was true, relevant, and unambiguous. However;, we cannot 
explain the subsequent failures to. provide the correct line numbers 
in terms of a violation of this contractual agreement between writers 
and readers because the students were->.expl i ci tl y informed that the 
"contract** had been violated. ^-^v 

Two alternative explanations are possible. Perhaps the nondetecting 
students were able to construct a consistent interpretation of the text, 
fitting the inconsistent information into what seemed to them a plausible 
schema (Collins et al., 1980; Rumelhart, 1980). But can we say that they 
actually comprehended the text? This seems instead to be a comprehension 
failure arising from misunderstanding; the reader feels satisfied with hi 
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Interpretation of the text, but it is not the'^one the author Intended tvo 
convey. 

Another possible reason why students did not report the Intended 
incohsisteacles is that they relied on a standard of externa i consistency; 
that is, they eval uated the information with respect t/j' thei r prior 
knowledge. Support for this explanation is provided both in Baker (1979a) 
and In the validation study reported here. Mar?;/ subjects who did not 
report the intended Inconsistency did in fa^'t report that specific infor- 
nation within the passage was i neons ist^^nt with what they believed to 
be true. For example, in the sampl^ passage shown In Table 1, a number 
of subjects noted that Moslem scrciety did not Include serfs and poor 
peasants. This tendency to/<4port conf 1 1 cts wi th prior knowledge was 
particularly pronounced^i:^n the passages did not contain experimenter- 
introduced confuslonf/. (Recall that only 23% of the subjects in the 



validation study ^/orrectly indicated that no Inconsistencies were 
present.) jy/s suggests that the first standard many students applied 
when eval w'a ting the passages was one of internal consistency ; If no 
.Interpol inconsistency was found, they proceeded to test for external 
cprisistency . The nature of the criteria students apply when eval uating 
'their comprehension warrants further investigation, especial ly in view 
of the' evidence that there is a developmental shift from overrel lance 
on external standards to effecti ve-'use of internal standards (Markman, 
1979; Osherson S Markrrian, 1975). 
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A secondary purpose of this study was to develop a profile of the 
successful comprehension monitor- We hoped to identify individuals who 
exhibited consistency In their. processing of different passages and who 
successfully Identified all inconsistencies- Howeyer> we found that, 
only 16^ of the subjects spent more time reading the' inconsi stent versions 
of both the main point and detail target sentences relative to the amount 
of time they spent on consistent target sentences- Efforts to identify 
consistent users pf the Immediate lookback strategy were also unsuccessful; 
only three subjects, reread the sentences preceding the main point and 
detail targets When, and only when, the targets were inconsistent. Even 
with respect to confusion detection, only 25% of the subjects provided 
the correct line numbers for all three types of inconsistencies- Jn 
short, our attempt to characterize good comprehension monitoring in 
terms of individual consistency did not succeed. 

Similarly, we were unable to discern systematic relationships be- 
tween confusion identification and reading behavior- For example, sub- 
jects who spent more time on the inconsistent target sentences during 
reading were no more ,lii<e1y to provide the correct line numbers than 
those who did not. All we can say is that subjects who spent more time 
on the inconsistent sentences aiiid_ provided the correct line numbers 
were more likely to report noticing the confusion during reading rather 
than after. (For the main point inconsistency, this was true for 23 
of 30 subjects; for the detail inconsistency, 19 of 22.) Thus, in- 
creases in "reading time reflect awareness of inconsistencies, but the . 
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cpiiverse^is not true; the absence of longer rea^^ng times does not indi 
/cate that/ subjects failed to notice the disruption^. 

These results are analogous to Rothkopf and Bi i ngton *s (1979) 
observation that, .despl te large .individual differences^ in processing 

strategies during reading, subjects attained the same goals. . The impl i 

' - ■ - ■ ' ' ' \' " 

cation Is that attempting to specify characteristics of successful 

comprehension monitor is the wrong approach; there is no single most 

effective processing style. Readers have a wide variety of monitoring 

activities available to them, and these can be used with flexibility 

. and effectiveness. 
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- V Footnote 

Hhe wording of this option was deliberately vague to avoid cluing 
subjects that inconsistencies were present. The "wording, however, may 
have been too vague, since few subjects selected this option (see Table 
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Table 1 . •• . 
/ Sample Paragraph",and Questions - 

The early Moslem society is best characterized by great affluence 
(poverty^). The Moslems control led the trade routes and had extensive 
trade throughout the Icnown world. They also had many highly successful 
industries, all of which were privately owned and operated. The textile 
industry was especially wealthy due to independent (government'^) owner- 
ship. Agriculture flourished, with a great, variety of commodities 

J 

produced by serfs and poor peasants. 

Main Point Question - ^ Detail Question 

Early Moslem economy was ' . - . T^e textile Industry' was owned by 



1 . 


very poor. 


1. 


the publ 1 c sector. 


2. 


highly successful. \ 


2. 


thfe private sector. 


3. 


stricken with bad management. 


3. 


the church. 


h. 


at the mercy of the sea winds. 


A. 


the cal 1 ph. 


5. 


cannot be answered on the basi s 


5. 


Cannot be answered on the 




of the paragraph. ' 




basis of the paragraph. 



^Mal n poi ntM neons I stency 
'^Detail Inconsistency 
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, Table 2 
Mean Total Exposure Times (sec) 
and Number of Re-exposures of Target Sentences 



Type ' 
Inconsistency 



.Main Point 
Detal 1 
None 



Exposure Time (sec) 



Number of Re-exposures 



Main Point 
Sentence 



6,80 
7.39 



Detail , 
Sentence 



Main Point 
Sentence 



7-80 
9.21 
6.26 



.90 
.51 
.57 



^ Detai 1 
Sentence 



.58 
.58 
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^ ■ Table 3 
Proportion of Responses in Each Category 
and Response Times on Target Questions 



Question 

-Main Point Detail 



\ 





Main Point 


Detai 1 


None 


Main Point 


Detai 1 


None 


Response Category 














Cons istent 


.32 


.80 


. .86 


.79 


.33 


.82 


Inconsistent 


.^1 


. .01 


.01 


.07 




.08 . 


**Can*t answer'* 


.16 


.07 


.10 


.12 


.18 


.10 


Other 


.11 


.12 


.Ok 


.03. 


.03 


.00 


Response Time (sec) 17.03 


12.38 


12.62" 


13.10 


I6.5A 


13.19 
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Table k 

Proportion of Correct Identifications of Inconsistencies 



Type of Inconsistency 





Main Point 


Deta i 1 


None 


Main Experiment 




.62 


.68 


Validation Study 








Line Numbers 




.61 


.27 


Explanations 


.79 


• 59 . 
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Appendix A / 
Summary of Significant Statistical Tests 
Involving Order and Passage Effects 

Reading Times / 

■ • ' /' 
• / 

Main Point Sentences / 

■ / , . 

Order: ^ £(2,162) = 7/2^ 
Passage: F(2J62)/= 5.79 



/ 

/' 

Detail Sentences ,/ 



Order: r(2,l62). = 32. 3Q 

■ ,/ ■ ' , ■■ ' 

Passage: \F.(2, 1 62). = 6.73 

Order X Passage: F(2,l62). = 3.39 
Order X Type of, Inconsistency: £(2,162) = M.OJ 
y'. Passage X Type of Inconsistency: £(.2,162) = 12,95 

Order X Passage X Type of Inconsistency: £(2,162) = 1^'^!.? 
/ Number of Re-exposures ^ . - ' 

Detail Sentences " ^ 

Order: F(2,l62) =6.73 

Order X Passage X Type of Inconsistency: £(2,t62).= 3»05 
Proportion of Consistent Responses on Target Questions 
Main Point Questions 

Passage: F(2,162) = 19.17 . 
Order X Type of Inconsistency: F(2,162) = A. 30 
Detai 1 Questions 
• Passage: K2,162) = 19. 6^* 
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Response Times on Target Questions 

Main Point Questions ■ ''SS^ 
Order: F(2,162) = A. 68 
Passage: F(2,162) =8.28 
Detai 1 Questions 

Passage: £.(2,162) = 5-38 
Identification of inconsistencies 

Order: f(2J62) = 5.17 
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Appendix B ' 
Interpretations and Means of Order Effects 

The purpose of this appendix Is to support our claim that effects of 
order are essential ly a result of greater fami 1 larl ty wi th the computer 
equipment and task demands. . As can be seen in tab.les 1 through ^ below, 
subjects spent more time on target sentences in initial passages than in 
later passages, and they re-exposed the detail target sentences more often; 
they answered the comprehension questions more slowly at first; and the 
probability of identifying inconsistencies was lower. 

There were two reliable interactions' of order with type of 
inconsistency, and the cell means are shown in Tables 5 and 6 below. The 
detail exposure time interaction (Table 5) shows decreasing times with 
Increasing order when inconsistencies were present. but equally short exposure 
times across order when no disruptions were present. The main point question 
interaction (Table. .6) is uninterpretable. The two triple interactions (not 
shown) are likewise uninterpretable. 

Table 1 

Exposure Times (sec) on Target Sentences 
' ' , ' First 

Main Point Target 9.37 
Detai 1 Target 9-56 

ERIC 



Second ' Third- 
7M 8.07 
7.ifl 6.30 
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Table 2 

Number of Re-Exposures of Detail Target 

Fl rst Second Thi rd 

. .78 .67, .46 

Table 3 

Response Times (sec), on Main Point Question 

Fi rst Second Thi rd 

15.78 13.97- 12.29 

Table A 

Proportion of Correct Inconsistency Identifications 

First Second Third 

.63 .5V -78 

./ ■ . ■ ■ • - 

Table 5 

Exposure Times cn Detail Targets as a 
Function of Order and Type of Inconsistency 
! First Second Third 

Main Point - ' 10.80 7.00 5.60 

Detail '. 11.^0 9.23 " 7.00 

None / 6.i»7 6.00 6.30 
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Table 6 

Proportion of "Consistent" Responses on Main Point Questions 
as a Function of Order and Type of Inconsl stency_ 

Fl rst Second Thl rd 

Main Point -33 -33 -27 

Detail '.. .80 .67 .90 ; 

None. .77 .93 , . -87 
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